{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding RNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What we use till now.\n",
    "Neural network that we use till now like dense, convnets have no memory.\n",
    "\n",
    "They were independent and no state were kept it between there input.\n",
    "\n",
    "With such network, to process sequence or series data we have to show entire data at once like we did with imdb.\n",
    "\n",
    "An entire movie review was transformed into a single large vector and processed in one go. Such networks are called\n",
    "feedforward networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recurrent neural network\n",
    "\n",
    "A recurrent neural network (RNN) processes sequences and keep them in memmory\n",
    "\n",
    "It works in batches\n",
    "\n",
    "We use it on sequentioal data where future is dependent on past\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# RNN Example : Pseudocode RNN\n",
    "\n",
    "#state_t = 0                          # The state at t\n",
    "#for input_t in input_sequence:       # Iterates over sequence elements\n",
    "#    output_t = f(input_t, state_t)   \n",
    "#    state_t = output_t               # The previous output becomes the state for the next iteration.\n",
    "\n",
    "# Another RNN Example : More detailed pseudocode for the RNN\n",
    "\n",
    "#state_t = 0\n",
    "#for input_t in input_sequence:\n",
    "#    output_t = activation(dot(W, input_t) + dot(U, state_t) + b)\n",
    "#    state_t = output_t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SimpleRNN\n",
    "\n",
    "SimpleRNN takes inputs of shape (batch_size, timesteps, input_features), rather than (timesteps,input_features)\n",
    "\n",
    "SimpleRNN can be run in two different modes:\n",
    "1. (a 3D tensor of shape (batch_size, timesteps, output_features))\n",
    "2. (a 2D tensor of shape (batch_size, output_features))\n",
    "\n",
    "These two modes are controlled by the return_sequences constructor argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding (Embedding)        (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn (SimpleRNN)       (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# example that uses SimpleRNN and returns only the output at the last timestep\n",
    "# (a 2D tensor of shape (batch_size, output_features))\n",
    "\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_1 (SimpleRNN)     (None, None, 32)          2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# The following example returns the full state sequence:\n",
    "# (a 3D tensor of shape (batch_size, timesteps, output_features))\n",
    "\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It’s sometimes useful to stack several recurrent layers one after the other in order to\n",
    "increase the representational power of a network. In such a setup, you have to get all\n",
    "of the intermediate layers to return full sequence of outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_2 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_4 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_5 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 328,320\n",
      "Trainable params: 328,320\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Rnn on IMDB Movie Review Classification Problem\n",
    "Enough with the examples lets implement the rnn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000 train sequences\n",
      "25000 test sequences\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import imdb  # loading IMdb from keras\n",
    "max_features = 10000                        # Max word\n",
    "\n",
    "(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "print(len(input_train), 'train sequences')\n",
    "print(len(input_test), 'test sequences')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_train shape: (25000, 500)\n",
      "input_test shape: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.preprocessing import sequence\n",
    "maxlen = 500                # lenth of sentence\n",
    "\n",
    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
    "\n",
    "print('input_train shape:', input_train.shape)\n",
    "print('input_test shape:', input_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training the model with Embedding and SimpleRNN layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 61s 3ms/sample - loss: 0.6450 - acc: 0.6209 - val_loss: 0.4932 - val_acc: 0.7912\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.4036 - acc: 0.8325 - val_loss: 0.3926 - val_acc: 0.8322\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.2907 - acc: 0.8857 - val_loss: 0.4193 - val_acc: 0.8032\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.2333 - acc: 0.9119 - val_loss: 0.4286 - val_acc: 0.8158\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.1531 - acc: 0.9467 - val_loss: 0.5607 - val_acc: 0.7614\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 61s 3ms/sample - loss: 0.0992 - acc: 0.9657 - val_loss: 0.5132 - val_acc: 0.8014\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.0679 - acc: 0.9783 - val_loss: 0.5297 - val_acc: 0.8248\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.0371 - acc: 0.9891 - val_loss: 0.5824 - val_acc: 0.8362\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.0226 - acc: 0.9940 - val_loss: 0.6661 - val_acc: 0.8168\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.0202 - acc: 0.9941 - val_loss: 0.7088 - val_acc: 0.8262\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))     # max_features = 10000\n",
    "model.add(SimpleRNN(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "\n",
    "history = model.fit(input_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plotting results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxV1bn/8c/DJAKCTLbKFFQqKBCIEbSioiiiFrBqFcQqTjgUemtrvVbsD6/W6q3WqbUqtVqtUcrVWtHrcB1wah0ICmEqg4gYoBgGQQGBwPP7Y+0kJyHDCZzknGy+79frvHLO3mvv/Zyd5DnrrL32WubuiIhIfDVKdwAiIlK3lOhFRGJOiV5EJOaU6EVEYk6JXkQk5pToRURiTol+L2Rmjc3sazPrmsqy6WRmh5pZyvsKm9nJZrYs4fVCMzsumbK7cayHzeyG3d1epCpN0h2A1MzMvk542QLYCuyIXl/h7nm12Z+77wBapbrs3sDdD0vFfszsMuACdx+csO/LUrFvkYqU6BsAdy9NtFGN8TJ3f62q8mbWxN2L6yM2kZro7zH91HQTA2b2KzP7q5k9ZWZfAReY2TFm9r6ZfWlmq8zsPjNrGpVvYmZuZlnR6yei9S+Z2Vdm9p6Zda9t2Wj9aWa2yMw2mNnvzOwfZja2iriTifEKM1tiZuvN7L6EbRub2d1mttbMPgGGVXN+bjSzKRWW3W9md0XPLzOzBdH7+SSqbVe1r0IzGxw9b2Fmf4limwccWclxl0b7nWdmI6LlfYDfA8dFzWJrEs7tTQnbXxm997Vm9nczOzCZc1Ob81wSj5m9ZmbrzOzfZnZdwnF+GZ2TjWaWb2YHVdZMZmbvlvyeo/P5dnScdcCNZtbDzKZH72VNdN7aJGzfLXqPRdH6e82seRRzr4RyB5rZZjNrX9X7lUq4ux4N6AEsA06usOxXwDZgOOHDe1/gKGAg4VvbwcAiYHxUvgngQFb0+glgDZALNAX+CjyxG2UPAL4CRkbrfgpsB8ZW8V6SifE5oA2QBawree/AeGAe0BloD7wd/pwrPc7BwNdAy4R9fwHkRq+HR2UMOAnYAvSN1p0MLEvYVyEwOHp+J/Am0BboBsyvUPZc4MDod3J+FMO3onWXAW9WiPMJ4Kbo+dAoxn5Ac+APwBvJnJtanuc2wGrgP4B9gNbAgGjdL4DZQI/oPfQD2gGHVjzXwLslv+fovRUDVwGNCX+P3wGGAM2iv5N/AHcmvJ+50flsGZU/Nlo3Gbg14Tg/A55N9/9hQ3ukPQA9avkLqzrRv1HDdtcC/xM9ryx5P5hQdgQwdzfKXgK8k7DOgFVUkeiTjPHohPV/A66Nnr9NaMIqWXd6xeRTYd/vA+dHz08DFlVT9gXgR9Hz6hL98sTfBXB1YtlK9jsXOCN6XlOifwz4dcK61oTrMp1rOje1PM8/BPKrKPdJSbwVlieT6JfWEMM5wIzo+XHAv4HGlZQ7FvgUsOj1LOCsVP9fxf2hppv4+DzxhZn1NLP/jb6KbwRuBjpUs/2/E55vpvoLsFWVPSgxDg//mYVV7STJGJM6FvBZNfECPAmMjp6fD5RewDaz75nZB1HTxZeE2nR156rEgdXFYGZjzWx21PzwJdAzyf1CeH+l+3P3jcB6oFNCmaR+ZzWc5y7Akipi6EJI9ruj4t/jt81sqpmtiGL4c4UYlnm48F+Ou/+D8O1gkJn1BroC/7ubMe21lOjjo2LXwocINchD3b018P8INey6tIpQ4wTAzIzyiamiPYlxFSFBlKip++dfgZPNrDOhaenJKMZ9gaeB2wjNKvsD/5dkHP+uKgYzOxh4gNB80T7a778S9ltTV9CVhOagkv3tR2giWpFEXBVVd54/Bw6pYruq1m2KYmqRsOzbFcpUfH//Tegt1ieKYWyFGLqZWeMq4ngcuIDw7WOqu2+topxUQYk+vvYDNgCbootZV9TDMV8AcsxsuJk1IbT7dqyjGKcCPzGzTtGFuf+srrC7ryY0LzwKLHT3xdGqfQjtxkXADjP7HqEtOdkYbjCz/S3cZzA+YV0rQrIrInzmXUao0ZdYDXROvChawVPApWbW18z2IXwQvePuVX5DqkZ153ka0NXMxptZMzNrbWYDonUPA78ys0Ms6Gdm7QgfcP8mXPRvbGbjSPhQqiaGTcAGM+tCaD4q8R6wFvi1hQvc+5rZsQnr/0Jo6jmfkPSllpTo4+tnwEWEi6MPEWq0dSpKpucBdxH+cQ8BPibU5FId4wPA68AcYAahVl6TJwlt7k8mxPwlcA3wLOGC5jmED6xkTCJ8s1gGvERCEnL3AuA+4MOoTE/gg4RtXwUWA6vNLLEJpmT7lwlNLM9G23cFxiQZV0VVnmd33wCcApxNuPi7CDghWn0H8HfCed5IuDDaPGqSuxy4gXBh/tAK760yk4ABhA+cacAzCTEUA98DehFq98sJv4eS9csIv+dt7v7PWr53oewCh0jKRV/FVwLnuPs76Y5HGi4ze5xwgfemdMfSEOmGKUkpMxtG+Cr+DaF7XjGhViuyW6LrHSOBPumOpaFS042k2iBgKeEr/TDgTF08k91lZrcR+vL/2t2XpzuehkpNNyIiMacavYhIzGVcG32HDh08Kysr3WGIiDQoM2fOXOPulXZnzrhEn5WVRX5+frrDEBFpUMysyrvD1XQjIhJzSvQiIjFXY6I3s0fM7Aszm1vFeovGnV5iZgVmlpOw7iIzWxw9Lkpl4CIikpxk2uj/TJgkoaoxJk4jjFfdgzDm9QPAwGhMjEmEccsdmGlm09x9fW2D3L59O4WFhXzzzTe13VTqUfPmzencuTNNm1Y1fIuIpEONid7d37ZodqEqjAQej8a/eD8a4OlAYDDwqruvAzCzVwk30DxV2yALCwvZb7/9yMrKIgyIKJnG3Vm7di2FhYV079695g1EpN6koo2+E+XHni6MllW1fBdmNi6apiy/qKhol/XffPMN7du3V5LPYGZG+/bt9a1LGpS8PMjKgkaNws+8vJq2aJhxpKJ7ZWXZ16tZvutC98mEkfHIzc2ttIySfObT70gakrw8GDcONm8Orz/7LLwGGLO744RmaBypqNEXUn7yhc6EEQurWi4iknYTJ5Yl1xKbN4flcYsjFYl+GnBh1PvmaGCDu68CXgGGmllbM2tLmJ7tlRQcr96tXbuWfv360a9fP7797W/TqVOn0tfbtm1Lah8XX3wxCxcurLbM/fffT166vjuK1KNMaDJZXsUQaVUtb9Bx1DSpLOHi6SpgO6GWfilwJXBltN6A+wlzS84BchO2vYQwH+US4OJkJrE98sgjvaL58+fvsqw6Tzzh3q2bu1n4+cQTtdq8WpMmTfI77rhjl+U7d+70HTt2pO5ADVRtf1ey93niCfcWLdyh7NGiRWr/T5PRrVv5GEoe3bo1zDioYpJ3T2ZycHcf7e4HuntTd+/s7n9y9wfd/cFovbv7j9z9EHfv4+75Cds+4u6HRo9HU/XhVJ2S9q7PPgunq6S9qy5qDEuWLKF3795ceeWV5OTksGrVKsaNG0dubi5HHHEEN998c2nZQYMGMWvWLIqLi9l///25/vrryc7O5phjjuGLL74A4MYbb+See+4pLX/99dczYMAADjvsMP75zzCxzqZNmzj77LPJzs5m9OjR5ObmMmvWrF1imzRpEkcddVRpfB6NUrpo0SJOOukksrOzycnJYdmyZQD8+te/pk+fPmRnZzOxvr+7yl4lU5pMbr0VWrQov6xFi7A8dnFU9QmQrsee1ujr+lM6sUa/ePFiNzP/8MMPS9evXbvW3d23b9/ugwYN8nnz5rm7+7HHHusff/yxb9++3QF/8cUX3d39mmuu8dtuu83d3SdOnOh33313afnrrrvO3d2fe+45P/XUU93d/bbbbvOrr77a3d1nzZrljRo18o8//niXOEvi2Llzp48aNar0eDk5OT5t2jR3d9+yZYtv2rTJp02b5oMGDfLNmzeX23Z3qEYvNTGr/H/UrP5jqctv//UdB3tSo29o6rvd7ZBDDuGoo44qff3UU0+Rk5NDTk4OCxYsYP78+btss++++3LaaacBcOSRR5bWqis666yzdinz7rvvMmrUKACys7M54ogjKt329ddfZ8CAAWRnZ/PWW28xb9481q9fz5o1axg+fDgQbnBq0aIFr732Gpdccgn77rsvAO3atav9iRBJUteutVtel8aMgWXLYOfO8LM+e9vUZxyxS/T1/UfUsmXL0ueLFy/m3nvv5Y033qCgoIBhw4ZV2q+8WbNmpc8bN25McXFxpfveZ599dinjSUwUs3nzZsaPH8+zzz5LQUEBl1xySWkclXWBdHd1jZR6kylNJnuT2CX6dP4Rbdy4kf3224/WrVuzatUqXnkl9Z2MBg0axNSpUwGYM2dOpd8YtmzZQqNGjejQoQNfffUVzzzzDABt27alQ4cOPP/880C4EW3z5s0MHTqUP/3pT2zZsgWAdevWpTxuyRzp7vEyZgxMngzduoFZ+Dl5cvpq03uDjBuPfk+V/LFMnBiaa7p2DUm+Pv6IcnJyOPzww+nduzcHH3wwxx57bMqPMWHCBC688EL69u1LTk4OvXv3pk2bNuXKtG/fnosuuojevXvTrVs3Bg4cWLouLy+PK664gokTJ9KsWTOeeeYZvve97zF79mxyc3Np2rQpw4cP55Zbbkl57JJ+mXKT0JgxSuz1KePmjM3NzfWKE48sWLCAXr16pSmizFJcXExxcTHNmzdn8eLFDB06lMWLF9OkSWZ8Zut3ldmyskJyr6hbt9A2LA2Xmc1099zK1mVGdpCkff311wwZMoTi4mLcnYceeihjkrxkvky5SUjqlzJEA7P//vszc+bMdIchDVTXrpXX6NPR40XqT+wuxopI1dTjZe+kRC9ST9Ld2wXU42VvpaYbkXqQKb1dSo6nxL53UY1epB5kyvgusndSok/C4MGDd7n56Z577uHqq6+udrtWrVoBsHLlSs4555wq912xO2lF99xzD5sTssTpp5/Ol19+mUzokiHU20XSSYk+CaNHj2bKlCnllk2ZMoXRo0cntf1BBx3E008/vdvHr5joX3zxRfbff//d3p/Uv0wa30X2Pkr0STjnnHN44YUX2Lp1KwDLli1j5cqVDBo0qLRfe05ODn369OG5557bZftly5bRu3dvIAxPMGrUKPr27ct5551XOuwAwFVXXVU6xPGkSZMAuO+++1i5ciUnnngiJ554IgBZWVmsWbMGgLvuuovevXvTu3fv0iGOly1bRq9evbj88ss54ogjGDp0aLnjlHj++ecZOHAg/fv35+STT2b16tVA6Kt/8cUX06dPH/r27Vs6hMLLL79MTk4O2dnZDBkyJCXndm+h3i6STg3uYuxPfgKVDL++R/r1gyhHVqp9+/YMGDCAl19+mZEjRzJlyhTOO+88zIzmzZvz7LPP0rp1a9asWcPRRx/NiBEjqhwk7IEHHqBFixYUFBRQUFBATk5O6bpbb72Vdu3asWPHDoYMGUJBQQE//vGPueuuu5g+fTodOnQot6+ZM2fy6KOP8sEHH+DuDBw4kBNOOIG2bduyePFinnrqKf74xz9y7rnn8swzz3DBBReU237QoEG8//77mBkPP/wwv/nNb/jtb3/LLbfcQps2bZgzZw4A69evp6ioiMsvv5y3336b7t27azycWkrn0BwiqtEnKbH5JrHZxt254YYb6Nu3LyeffDIrVqworRlX5u233y5NuH379qVv376l66ZOnUpOTg79+/dn3rx5lQ5Ylujdd9/l+9//Pi1btqRVq1acddZZvPPOOwB0796dfv36AVUPhVxYWMipp55Knz59uOOOO5g3bx4Ar732Gj/60Y9Ky7Vt25b333+f448/nu7duwMaynh3ZMqQuLL3SapGb2bDgHuBxsDD7n57hfXdgEeAjsA64AJ3L4zW7SBMMQiw3N1H7EnA1dW869KZZ57JT3/6Uz766CO2bNlSWhPPy8ujqKiImTNn0rRpU7KysiodmjhRZbX9Tz/9lDvvvJMZM2bQtm1bxo4dW+N+qhunqGSIYwjDHFfWdDNhwgR++tOfMmLECN58801uuumm0v1WjFFDGYs0XDXW6M2sMWFO2NOAw4HRZnZ4hWJ3Ao+7e1/gZuC2hHVb3L1f9NijJJ9OrVq1YvDgwVxyySXlLsJu2LCBAw44gKZNmzJ9+nQ+q+z+8gTHH3986QTgc+fOpaCgAAhDHLds2ZI2bdqwevVqXnrppdJt9ttvP7766qtK9/X3v/+dzZs3s2nTJp599lmOO+64pN/Thg0b6NSpEwCPPfZY6fKhQ4fy+9//vvT1+vXrOeaYY3jrrbf49NNPAQ1lLNKQJNN0MwBY4u5L3X0bMAUYWaHM4cDr0fPplayPhdGjRzN79uzSGZ4AxowZQ35+Prm5ueTl5dGzZ89q93HVVVfx9ddf07dvX37zm98wYMAAIMwW1b9/f4444gguueSSckMcjxs3jtNOO630YmyJnJwcxo4dy4ABAxg4cCCXXXYZ/fv3T/r93HTTTfzgBz/guOOOK9f+f+ONN7J+/Xp69+5NdnY206dPp2PHjkyePJmzzjqL7OxszjvvvKSPk26ZcEeqSDrVOEyxmZ0DDHP3y6LXPwQGuvv4hDJPAh+4+71mdhbwDNDB3deaWTEwCygGbnf3v1d3PA1T3LBl2u+q4h2pEHq76LZ/iZvqhilOpkZfWcNsxU+Ha4ETzOxj4ARgBSGxA3SNDn4+cI+ZHVJJgOPMLN/M8ouKipIISSQ5uiNVJLlEXwh0SXjdGViZWMDdV7r7We7eH5gYLdtQsi76uRR4E9ilbcHdJ7t7rrvnduzYcXfeh0ildEeqSHKJfgbQw8y6m1kzYBQwLbGAmXUws5J9/YLQAwcza2tm+5SUAY4Fqu8zWIVMmwlLdpWJvyPdkSqSRKJ392JgPPAKsACY6u7zzOxmMyvpRTMYWGhmi4BvASX3+/UC8s1sNuEi7e3uXutE37x5c9auXZuRiUQCd2ft2rU0b9483aGUoztSRRrInLHbt2+nsLCwxn7lkl7Nmzenc+fONG3aNN2hlJOXpztSJf6quxjbIBK9iIhUb0973YiISAOmRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzSvQiIjGnRC8iEnNK9CIiMadELyISc0r0IiIxp0QvdUZT+IlkhibpDkDiqeIUfp99Fl6DRo4UqW+q0Uud0BR+IplDiV7qhKbwE8kcSvRSJzSFn0jmUKKXOqEp/EQyR1KJ3syGmdlCM1tiZtdXsr6bmb1uZgVm9qaZdU5Yd5GZLY4eF6UyeMlcY8bA5MnQrRuYhZ+TJ+tCrEg61DiVoJk1BhYBpwCFwAxgdOIk32b2P8AL7v6YmZ0EXOzuPzSzdkA+kAs4MBM40t3XV3U8TSUoIlJ7ezqV4ABgibsvdfdtwBRgZIUyhwOvR8+nJ6w/FXjV3ddFyf1VYFht34CIiOy+ZBJ9J+DzhNeF0bJEs4Gzo+ffB/Yzs/ZJbouZjTOzfDPLLyoqSjZ2ERFJQjKJ3ipZVrG951rgBDP7GDgBWAEUJ7kt7j7Z3XPdPbdjx45JhCQiIslK5s7YQqBLwuvOwMrEAu6+EjgLwMxaAWe7+wYzKwQGV9j2zT2IV0REaimZGv0MoIeZdTezZsAoYFpiATPrYGYl+/oF8Ej0/BVgqJm1NbO2wNBomYiI1JMaE727FwPjCQl6ATDV3eeZ2c1mNiIqNhhYaGaLgG8Bt0bbrgNuIXxYzABujpaJiEg9qbF7ZX1T98o9l5cXxpRZvjzciXrrreq/LhJ31XWv1OiVMaNRI0WkIg2BEDMaNVJEKlKijxmNGikiFSnRx4xGjRSRipToY0ajRopIRUr0MaNRI0WkIvW6iaExY5TYRaSMavQiIjGnRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzSvQiIjGnRC8iEnNK9CIiMadELyISc0klejMbZmYLzWyJmV1fyfquZjbdzD42swIzOz1anmVmW8xsVvR4MNVvQEREqlfjWDdm1hi4HzgFKARmmNk0d5+fUOxGwlyyD5jZ4cCLQFa07hN375fasEVEJFnJ1OgHAEvcfam7bwOmACMrlHGgdfS8DbAydSGKiMieSCbRdwI+T3hdGC1LdBNwgZkVEmrzExLWdY+adN4ys+P2JFgREam9ZBK9VbLMK7weDfzZ3TsDpwN/MbNGwCqgq7v3B34KPGlmrStsi5mNM7N8M8svKiqq3TsQEZFqJZPoC4EuCa87s2vTzKXAVAB3fw9oDnRw963uvjZaPhP4BPhOxQO4+2R3z3X33I4dO9b+XYiISJWSSfQzgB5m1t3MmgGjgGkVyiwHhgCYWS9Coi8ys47RxVzM7GCgB7A0VcGLiEjNaux14+7FZjYeeAVoDDzi7vPM7GYg392nAT8D/mhm1xCadca6u5vZ8cDNZlYM7ACudPd1dfZuRERkF+Zesbk9vXJzcz0/Pz/dYYiINChmNtPdcytbpztjRURiTok+hfLyICsLGjUKP/Py0h2RSObYsgXy8+HRR+Gaa2DIEDjiCLj6anj5Zdi6Nd0RxleNbfSSnLw8GDcONm8Orz/7LLwGGDMmfXGJ1LedO8Pff0FB2WPOHFi8OKwD2Hdf6N0bunaFxx6DBx6AVq3g1FNhxAg4/XTo0CG97yNO1EafIllZ4Y+7om7dYNmy+o5GpH58+WVI4iXJvOTn11+XlTnkEOjbNzz69Ak/Dz4YGjcO67/5Bt54A6ZNg+efh5Urw7fi7343JP0RI+Cww9Lz/hqS6trolehTxCq7rSyybh20bVt/sYik2vbtsGhRWTIvSejLl5eVadu2fDLv2zc0zbRqlfxx3OGjj0LSnzYNZs0Ky7/zHRg+PCT9734XmqgtYhdK9HXIHaZMgQsuKPtaWlHLlnDZZfCTn4Sav0imcofVq8sn84ICmD8ftm0LZZo0gZ49d62ld+pUfYVndyxfDi+8EJL+G2+ED5x27eCMM0LSHzoUWu9yr/3eSYm+jixfDlddBS++GL6erlgRvoaWaNECbrwRFiyAp54K/0Q/+AFcey0ceWT64haBcHF03rzytfSCAlizpqzMQQftmtB79oRmzeo/3o0b4f/+LzTvvPBC+KbctCmceGJI+sOHhzb/hsId1q4N1y4WLQqPNm3guut2b39K9Cm2cyf84Q/wi1+EX9att8L48aFmP3Fi+ADo2jUsL7kQW1gI990HDz0U/mAHD4af/xyGDQvtkSL1YckS+Mtf4Omn4V//2vXiaElSL0ns7dunN96qFBfDe++VNfEsWhSWZ2eXtevn5GTG/9ZXX4VknpjQS56vX19WrnFjOOUUeOml3TuOEn0KzZ8fmmHeey/0EHjwwdo1x2zcCA8/DPfcA59/DocfDj/7WfhA2GefOgtb9mLr18PUqfD44/DPf4bmlRNPhOOOq/ziaEO0cGGo6U+bBv/4R/gAO+igUMsfPhxOOil8mNWVrVvhk0/KJ/GS56tWlS/bpUu45vCd70CPHmXPs7LCN5TdpUSfAlu3wu23h1p669YhUY8Zs/ttktu3h3++O+6A2bPh29+GH/8YrrxSF25lz23bFvqmP/54SIDbtoVKxYUXhr/bzp3THWHdWbMmNKc+/3w4B19/HZpRhw4NNf0zzoADDqj9fnfsCD3rKtbKFy0K3+ITr9EdcED5JF7y/JBDQix1QYl+D733XqjFz58P558fknyqBtl0h9dfhzvvhFdeCRduL7003FCiC7dSG+4wc2ZI7k89FRJex47hb/bCC6F//9RfLM10W7fCm2+Wdd38/PNwDo4+uqyJp1evsvPiHmrglSXzTz4JFbQSrVuXT+YlCb1HD9h///p/r0r0u+mrr0Kb++9/H2pADz4YbuSoKwUFcNdd8OSTofZQcuE2t9JfXcNQXBzaSTOhrTSuPv8cnngiJPh//Ss0AY4cGZL70KF71hwQJ+7h23NJu/7MmWF5ST//Tz8NiX3TprJt9tmnLHlXrJ0fcEBmfXAq0e+Gl14KzSiffw4TJsCvfgX77Vc/x16xIly4ffDB0KZ/wgnhwu1pp2V2wty+PfTi+Oij8E/00UfhH6tjR3juOeinmYNT5quv4G9/C8l9+vSQxAYNCsn9Bz9IT42yoVmxIvTeee65UFs/9NBdE3qXLpn9P5dIib4WiopCf/cnnwxtmg8/DMcck55YKl647dWr7MJt8+bpianE1q0wd25ZQp85M3wjKelrvd9+oamgf/+QkNatC80Jw4enN+6GbMeO0Mz3+OPhnG7ZEmqjF14Y7uM4+OB0RyjpVF2ix90z6nHkkUd6Ouzc6f6Xv7i3b+/etKn7TTe5f/NNWkLZxbZt7nl57v36uYP7t77l/qtfua9dWz/H37zZ/YMP3P/wB/fLLnPv3z+co1CPdG/Txv3EE92vvdb9ySfdFy5037GjbPuVK91zc93N3O++O5xrSV5BgfvPf+5+0EHhfO+/v/sVV7j/4x86l1KGMD9IpXk17Ym94iMdiX7ZMvdTTw1n45hj3OfNq/cQkrJzp/trr7kPGxZibdHCfcIE96VLU3eMTZtCAvnd79zHjnXv29e9ceOypN6unfspp7j/53+6T53qvmRJcslm0yb3s84K+7j6avft21MXcxz9+9/ud91V9uHepIn7iBHuTz/tvmVLuqOTTKREX4XiYvd77nFv2dK9VauQ3BJropmsoCAk4qZN3Rs1cj/3XPcPP6zdPjZudH/77XAOfvhD98MPD/sqSeodO4YPlYkT3Z95Jnwg7kkNcseO8AEB4YN1w4bd31ccbd7sPmWK++mnl324HnVU+Lv84ot0RyeZrrpEv9e20c+dG7pMfvBB6EnzwAMN6/bpEitWwO9+Fy7cbtgAxx8feuqccUb5i0gbNsDHH5dvU1+0KKR0gAMPDMMy5OSU/ayLsUsA/vSncKG7Z89wMaxbt9Qfo6HYuRPefTe0u//P/4TrMp07ww9/GB69eqU7Qmko9riNHhgGLASWANdXsr4rMB34GCgATk9Y94tou4XAqTUdq65r9N984/7LX4avwh06hDblOLRzbtwY2r+7dg01wZ493W+8MdT0Dz20rJYO7l26uGRahmoAAA6kSURBVI8c6X7zze4vvBDa0Ovb66+HtuYDDnB///36P366LVoU/g6zssLvpFWr8A3t9dcbzrdKySzsSdMNYULwT4CDgWbAbODwCmUmA1dFzw8HliU8nw3sA3SP9tO4uuPVZaJ/992QACE0VRQV1dmh0mbbtvDh1b9/eJ9ZWe5nn+1+663uL7/svnp1uiMss2CB+8EHuzdvHtr746642P3hh8N1IAjNZEOHuj/xhPvXX6c7Omnoqkv0yYzqPABY4u5Lo68HU4CRwPzELwZAyWChbYCV0fORwBR33wp8amZLov29l8RxU2bjxjAA2R/+EJoJXn45jFMTR02bwujRMGpUmO2qZct0R1S1nj1D09mZZ8K558Kvfw3XX59ZN6GkykcfwRVXhKn0evcOQ1+cf34Yj0WkriVzK0An4POE14XRskQ3AReYWSHwIjChFttiZuPMLN/M8ouKipIMPTkvvBAmP3jggdA/fu7c+Cb5RGaZneRLdOgAr70Wkt4NN8All5T1xY+Dr78O9z4cdVS4F2LKlHC/wbXXKslL/Ukm0VdWv6p4BXc08Gd37wycDvzFzBoluS3uPtndc909t2OKBpFZvTrUaocPD3cJvvce3H137Wa7kfrRvHm4hf+mm+DPfw637a9bl+6o9tzzz4eb7u66Cy6/PAxPcN558fzGIpktmURfCHRJeN2ZsqaZEpcCUwHc/T2gOdAhyW1Tyj1MNtyrFzz7LNxyS+hhMnBgXR5V9pQZTJoUEv5774W7kRcvTndUu2fFCjjnnDBgVuvWYdjcBx/UsASSPskk+hlADzPrbmbNgFHAtApllgNDAMysFyHRF0XlRpnZPmbWHegBfJiq4Cv69NPQLDN2bGiumT07zPCUjtlwZPeMGROmjFu3Loww+M476Y4oeTt2wP33h0rG//5vuObw0UdhjlORdKox0bt7MTAeeAVYAEx193lmdrOZjYiK/Qy43MxmA08BY6MLwfMINf35wMvAj9x9R128kYULw0Wu998PF13feitc7JOG59hjw+/xgANgyJAwI1Kmmz07JPTx48MH1Ny5oQOAKhmSCWJzw5RHU/qNHRvvSRX2JuvXhyaQN96AX/4S/uu/Mq99e9OmENddd4VJq++5J/R6yrQ4Jf6qu2GqgQzAWTOz0EyjJB8fbduGrrCXXhqutZx/fvnJ19PtxRdDE+Edd8DFF4eLreefryQvmSc2iV7iqWlT+OMf4Te/CV0TTzoJvvgivTGtWhV6z5xxRpgW7u23Q4zt2qU3LpGqKNFLxjMLE6888wzMmhV6UM2fX/N2qbZzZ7gfo2fPMFnFLbeEeI47rv5jEakNJXppMM46K1xk/+ab0P3y1Vfr79hz5oSLxFdfHaZ2nDNHPbqk4VCilwblqKPCsAnduoWpFSdPrtvjbd4ces/k5MCSJWGUyddeC9PMiTQUSvTS4HTtGob2HTo0jB9z7bWhD3uqvfJK6LJ7++1hyOAFC8JPXWyVhkaJXhqk1q1h2rTQb/23v4Wzzw5dHVNh9erQe2bYsNA0M306PPJIGJdHpCFSopcGq0mTMOnKffeFcWWOPx5W7sEAGzt3ht4zPXuGC7833RRuhBo8OFURi6SHEr00eBMmhNr9okUwYEDoCVNb8+aFD4px4yA7OyT4SZNgn31SH69IfVOil1g444zQbm8GgwaF4amTsWVL6D3Tv39og3/00dBUo+EzJE6U6CU2srPhww9Dkh45Eu69t2xO3Mq89hr06ROGzhg9OtzZOnasLrZK/CjRS6wceGDoaz9yZJhoZvx4KC4uX+aLL0LvmVNOCROov/56GNo6RVMhiGQcJXqJnZYt4emn4brrwkimw4eH6STdQ++ZXr3gr38NA6UVFIRhFUTiLJk5Y0UanEaN4L//O9zYdNVV4a7Wdu3CuDSDBsFDD4XZn0T2Bkr0EmuXXQbdu4d+9itWwMMPh5EmG+m7rOxFlOgl9oYMCRPTNGsWhj4W2dso0cte4VvfSncEIumT1BdYMxtmZgvNbImZXV/J+rvNbFb0WGRmXyas25GwruJcsyIiUsdqrNGbWWPgfuAUoBCYYWbT3L10RHB3vyah/ASgf8Iutrh7v9SFLCIitZFMjX4AsMTdl7r7NmAKMLKa8qMJE4SLiEgGSCbRdwI+T3hdGC3bhZl1A7oDbyQsbm5m+Wb2vpmduduRiojIbknmYmxlN4RXdWP5KOBpd08cHbyru680s4OBN8xsjrt/Uu4AZuOAcQBdu3ZNIiQREUlWMjX6QqBLwuvOQFWDwY6iQrONu6+Mfi4F3qR8+31JmcnunuvuuR11H7qISEolk+hnAD3MrLuZNSMk8116z5jZYUBb4L2EZW3NbJ/oeQfgWCAN0zqLiOy9amy6cfdiMxsPvAI0Bh5x93lmdjOQ7+4lSX80MMW93HiBvYCHzGwn4UPl9sTeOiIiUvfMqxvHNQ1yc3M9Pz8/3WGIiDQoZjbT3XMrW6cRP0REYk6JXkQk5pToRURiToleRCTmlOhFRGJOiV5EJOaU6EVEYk6JXkQk5pToRURiToleRCTmlOhFRGJOiV5EJOaU6EVEYk6JXkQk5pToRURiToleRCTmlOhFRGJOiV5EJOaSSvRmNszMFprZEjO7vpL1d5vZrOixyMy+TFh3kZktjh4XpTJ4ERGpWY2Tg5tZY+B+4BSgEJhhZtMSJ/l292sSyk8A+kfP2wGTgFzAgZnRtutT+i5ERKRKydToBwBL3H2pu28DpgAjqyk/Gngqen4q8Kq7r4uS+6vAsD0JWEREaieZRN8J+DzhdWG0bBdm1g3oDrxRm23NbJyZ5ZtZflFRUTJxi4hIkpJJ9FbJMq+i7CjgaXffUZtt3X2yu+e6e27Hjh2TCElERJKVTKIvBLokvO4MrKyi7CjKmm1qu62IiNSBZBL9DKCHmXU3s2aEZD6tYiEzOwxoC7yXsPgVYKiZtTWztsDQaJmIiNSTGnvduHuxmY0nJOjGwCPuPs/Mbgby3b0k6Y8Gpri7J2y7zsxuIXxYANzs7utS+xZERKQ6lpCXM0Jubq7n5+enOwwRkQbFzGa6e25l63RnrIhIzCnRi4jEnBK9iEjMKdGLiMScEr2ISMwp0YuIxJwSvYhIzCnRi4jEnBK9iEjMKdGLiMScEr2ISMwp0YuIxJwSvYhIzCnRi4jEnBK9iEjMKdGLiMScEr2ISMwllejNbJiZLTSzJWZ2fRVlzjWz+WY2z8yeTFi+w8xmRY9d5poVEZG6VeOcsWbWGLgfOAUoBGaY2TR3n59QpgfwC+BYd19vZgck7GKLu/dLcdwiIpKkZGr0A4Al7r7U3bcBU4CRFcpcDtzv7usB3P2L1IYpIiK7K5lE3wn4POF1YbQs0XeA75jZP8zsfTMblrCuuZnlR8vPrOwAZjYuKpNfVFRUqzcgIiLVq7HpBrBKlnkl++kBDAY6A++YWW93/xLo6u4rzexg4A0zm+Pun5TbmftkYDJAbm5uxX2LiMgeSKZGXwh0SXjdGVhZSZnn3H27u38KLCQkftx9ZfRzKfAm0H8PYxYRkVpIJtHPAHqYWXczawaMAir2nvk7cCKAmXUgNOUsNbO2ZrZPwvJjgfmIiEi9qbHpxt2LzWw88ArQGHjE3eeZ2c1AvrtPi9YNNbP5wA7g5+6+1sy+CzxkZjsJHyq3J/bWERGRumfumdUknpub6/n5+ekOQ0SkQTGzme6eW9k63RkrIhJzSvQiIjGnRC8iEnNK9CIiMadELyISc0r0IiIxp0QvIhJzsUn0eXmQlQWNGoWfeXnpjkhEJDMkM6hZxsvLg3HjYPPm8Pqzz8JrgDFj0heXiEgmiEWNfuLEsiRfYvPmsFxEZG8Xi0S/fHntlouI7E1ikei7dq3dchGRvUksEv2tt0KLFuWXtWgRlouI7O1ikejHjIHJk6FbNzALPydP1oVYERGISa8bCEldiV1EZFexqNGLiEjVlOhFRGJOiV5EJOaU6EVEYk6JXkQk5jJucnAzKwI+S3cce6gDsCbdQWQQnY/ydD7K6FyUtyfno5u7d6xsRcYl+jgws/yqZmPfG+l8lKfzUUbnory6Oh9quhERiTklehGRmFOirxuT0x1AhtH5KE/no4zORXl1cj7URi8iEnOq0YuIxJwSvYhIzCnRp5CZdTGz6Wa2wMzmmdl/pDumdDOzxmb2sZm9kO5Y0s3M9jezp83sX9HfyDHpjimdzOya6P9krpk9ZWbN0x1TfTKzR8zsCzObm7CsnZm9amaLo59tU3EsJfrUKgZ+5u69gKOBH5nZ4WmOKd3+A1iQ7iAyxL3Ay+7eE8hmLz4vZtYJ+DGQ6+69gcbAqPRGVe/+DAyrsOx64HV37wG8Hr3eY0r0KeTuq9z9o+j5V4R/5E7pjSp9zKwzcAbwcLpjSTczaw0cD/wJwN23ufuX6Y0q7ZoA+5pZE6AFsDLN8dQrd38bWFdh8Ujgsej5Y8CZqTiWEn0dMbMsoD/wQXojSat7gOuAnekOJAMcDBQBj0ZNWQ+bWct0B5Uu7r4CuBNYDqwCNrj7/6U3qozwLXdfBaHiCByQip0q0dcBM2sFPAP8xN03pjuedDCz7wFfuPvMdMeSIZoAOcAD7t4f2ESKvpY3RFHb80igO3AQ0NLMLkhvVPGlRJ9iZtaUkOTz3P1v6Y4njY4FRpjZMmAKcJKZPZHekNKqECh095JveE8TEv/e6mTgU3cvcvftwN+A76Y5pkyw2swOBIh+fpGKnSrRp5CZGaENdoG735XueNLJ3X/h7p3dPYtwke0Nd99ra2zu/m/gczM7LFo0BJifxpDSbTlwtJm1iP5vhrAXX5xOMA24KHp+EfBcKnYam8nBM8SxwA+BOWY2K1p2g7u/mMaYJHNMAPLMrBmwFLg4zfGkjbt/YGZPAx8Reqt9zF42HIKZPQUMBjqYWSEwCbgdmGpmlxI+DH+QkmNpCAQRkXhT042ISMwp0YuIxJwSvYhIzCnRi4jEnBK9iEjMKdGLiMScEr2ISMz9f8l3MZ74FF6DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU9b3/8deHHQTBAhVlh2Jlkc2IGwqobcENSlHZ3IpFrQsuvReuclvlp71udS1VqZV6JYoUq2JF6a1i1VqBgBgFRCgCBhQDFURBIfD5/fGdwCRMkglZTmbm/Xw88sicc75z5jMT+Mz3fM93MXdHRERSX62oAxARkcqhhC4ikiaU0EVE0oQSuohImlBCFxFJE0roIiJpQgldEjKz2mb2lZm1q8yyUTKz75lZpffTNbMzzGxt3PZKMzslmbIH8VqPmdlNB/v8Us57m5n9sbLPK9WrTtQBSOUws6/iNhsB3wJ7YtuXu3t2ec7n7nuAxpVdNhO4+/cr4zxmdhkw1t0Hxp37sso4t6QnJfQ04e77EmqsBniZu/+tpPJmVsfdC6ojNhGpHmpyyRCxS+pnzOxpM9sOjDWzE83sHTPbamafmtmDZlY3Vr6OmbmZdYhtz4gdf9nMtpvZP82sY3nLxo4PMbOPzGybmT1kZv8ws0tKiDuZGC83s9Vm9oWZPRj33Npmdp+ZbTGzfwGDS/l8JpvZzGL7pprZvbHHl5nZitj7+Ves9lzSufLMbGDscSMzezIW2zLg2ASvuyZ23mVmdm5s/zHAb4FTYs1Zm+M+21vinn9F7L1vMbPnzeyIZD6bspjZsFg8W83sNTP7ftyxm8xso5l9aWYfxr3XE8xsSWz/JjO7O9nXk0ri7vpJsx9gLXBGsX23AbuAcwhf5A2B44DjCVdqnYCPgKtj5esADnSIbc8ANgNZQF3gGWDGQZT9LrAdGBo7dgOwG7ikhPeSTIwvAE2BDsC/C987cDWwDGgDNAfeCP/kE75OJ+Ar4JC4c38OZMW2z4mVMeA0YCfQM3bsDGBt3LnygIGxx/cArwOHAe2B5cXKng8cEfubjI7FcHjs2GXA68XinAHcEnv8w1iMvYEGwO+A15L5bBK8/9uAP8Yed43FcVrsb3RT7HOvC3QH1gGtYmU7Ap1ijxcBo2KPmwDHR/1/IdN+VEPPLG+5+4vuvtfdd7r7Indf4O4F7r4GmAYMKOX5s909x913A9mERFLesmcDS939hdix+wjJP6EkY/wfd9/m7msJybPwtc4H7nP3PHffAtxRyuusAT4gfNEA/ADY6u45seMvuvsaD14DXgUS3vgs5nzgNnf/wt3XEWrd8a87y90/jf1NniJ8GWclcV6AMcBj7r7U3b8BJgEDzKxNXJmSPpvSjATmuPtrsb/RHcChhC/WAsKXR/dYs93Hsc8OwhdzFzNr7u7b3X1Bku9DKokSemb5JH7DzI42s5fM7DMz+xKYArQo5fmfxT3eQek3Qksqe2R8HO7uhBptQknGmNRrEWqWpXkKGBV7PJrwRVQYx9lmtsDM/m1mWwm149I+q0JHlBaDmV1iZu/Fmja2AkcneV4I72/f+dz9S+ALoHVcmfL8zUo6717C36i1u68EbiT8HT6PNeG1ihW9FOgGrDSzhWZ2ZpLvQyqJEnpmKd5l71FCrfR77n4o8EtCk0JV+pTQBAKAmRlFE1BxFYnxU6Bt3HZZ3SqfAc6I1XCHEhI8ZtYQmA38D6E5pBnw1yTj+KykGMysE/AwcCXQPHbeD+POW1YXy42EZpzC8zUhNO1sSCKu8py3FuFvtgHA3We4+8mE5pbahM8Fd1/p7iMJzWq/AZ41swYVjEXKQQk9szUBtgFfm1lX4PJqeM2/AH3N7BwzqwNMAFpWUYyzgOvMrLWZNQcmllbY3TcBbwHTgZXuvip2qD5QD8gH9pjZ2cDp5YjhJjNrZqGf/tVxxxoTknY+4bvtMkINvdAmoE3hTeAEngbGmVlPM6tPSKxvunuJVzzliPlcMxsYe+3/INz3WGBmXc1sUOz1dsZ+9hDewIVm1iJWo98We297KxiLlIMSema7EbiY8J/1UUINtUrFkuYFwL3AFqAz8C6h33xlx/gwoa37fcINu9lJPOcpwk3Op+Ji3gpcDzxHuLE4gvDFlIxfEa4U1gIvA/8bd95c4EFgYazM0UB8u/P/AauATWYW33RS+PxXCE0fz8We347Qrl4h7r6M8Jk/TPiyGQycG2tPrw/cRbjv8RnhimBy7KlnAiss9KK6B7jA3XdVNB5JnoUmTJFomFltwiX+CHd/M+p4RFKZauhS7cxssJk1jV22/zeh58TCiMMSSXlK6BKF/sAawmX7YGCYu5fU5CIiSVKTi4hImlANXUQkTUQ2OVeLFi28Q4cOUb28iEhKWrx48WZ3T9jVN7KE3qFDB3JycqJ6eRGRlGRmJY54VpOLiEiaUEIXEUkTSugiImlCCV1EJE0ooYuIpAkldBGRNKGELiKSJpTQRUSqyapV8KtfwbJlVXP+pAYWmdlg4AHC6iSPufsdxY7fBwyKbTYCvhtbfUVEJKNt3gzPPANPPgkLFoAZtGoF3btX/muVmdBj81VPJSyamwcsMrM57r68sIy7Xx9X/hqgT+WHKiKSGr75Bl58MSTxl1+GggLo0QPuvBNGj4Y2bco+x8FIpobeD1hduLK3mc0krLe4vITyowirtIiIZIy9e+HNN0MSnz0btm2DI46ACRPgwguhV6+qjyGZhN6aoquW5wHHJypoZu0JC8e+VsLx8cB4gHbtylqvV0Sk5luxIiTx7GxYvx4OOQSGDw9J/LTToHbt6oslmYSeaGXzkiZRHwnMdvc9iQ66+zRgGkBWVpYmYheRlLRpEzz9NMyYAYsXQ61a8MMfwq9/DcOGhaQehWQSeh7QNm67DWENyERGAldVNCgRkZpmxw54/vmQxP/6V9izB/r0gXvvhVGjwo3OqCWT0BcBXcysI7CBkLRHFy9kZt8nrAD+z0qNUEQkInv2wPz5IYk/+yx89RW0bQv/8R8wdmzV9FSpiDITursXmNnVwDxCt8XH3X2ZmU0Bctx9TqzoKGCma007EUlxubkhiWdnw8aNcOihcP75oV381FNDE0tNFNmaollZWa4FLkSkptiwIbSLP/lkSOh16sCQIaEmfs450LBh1BEGZrbY3bMSHYtsxSIRkaht3w7PPReS+Kuvgjscfzw89BBccAG0TLjQW82lhC4iGaWgAP72t5DEn38+3Ozs2BEmTw618aOOijrCg6eELiIZ4YMP4A9/CM0qmzbBYYeFNvELL4STTgpD8lOdErqIpLWCgtA/fMqUMMjn7LNDTfzMM6F+/aijq1xK6CKStj7+OCTvt9+GMWPggQegefOoo6o6SugikpZmzICf/zw0pcyYERJ6uquhvSlFRA7O1q0heV94IfTsCUuXZkYyhxRL6NnZ0KFD6NTfoUPYFhEp9NZb0Lt3mH98yhR4/fXQgyVTpEyTS3Y2jB8fuhgBrFsXtiFzvn1FJLHdu0MC//WvQ2XvrbfghBOijqr6pUwN/eab9yfzQjt2hP0ikrlWr4b+/eG22+Cii0ITSyYmc0ihhL5+ffn2i0h6c4fp00MTy0cfhWaW6dOhSZOoI4tOyiT0ktbD0DoZIpnn3/8OQ/N/+lPIygpzr5x/ftRRRS9lEvrtt0OjRkX3NWoU9otI5pg/Pyzn9txzcMcdYQ6Wtm3Lfl4mSJmEPmYMTJsG7duHfqXt24dt3RAVyQy7dsGkSXD66aEy9847MHFi9S7xVtOlTC8XCMlbCVwk86xcCaNHw5Il8LOfwX33RbfMW02WMjV0Eck87uFKvE8fWLsW/vznsK1knlhK1dBFJHNs3gyXXQYvvABnnAFPPAFHHhl1VDWbaugiUuP89a9wzDHw8svwm9/AvHlK5slQQheRGuPbb+GGG+BHPwrzlS9YELZr6hqeNU1SH5OZDTazlWa22swmlVDmfDNbbmbLzOypyg1TRNLdsmXQr1+44XnVVZCTEwYNSfLKbEM3s9rAVOAHQB6wyMzmuPvyuDJdgP8CTnb3L8zsu1UVsIikF3f43e/gF78IozxffDEsQiHll0wNvR+w2t3XuPsuYCYwtFiZnwFT3f0LAHf/vHLDFJF0tGlTSN5XXw2DBsH77yuZV0QyCb018Encdl5sX7yjgKPM7B9m9o6ZDU50IjMbb2Y5ZpaTn59/cBGLSFqYOzfMV/7qq/DQQ/DSS3D44VFHldqSSeiJlk71Ytt1gC7AQGAU8JiZNTvgSe7T3D3L3bNatmxZ3lhFJA3s3AnXXANnnRUSeE5OqKGnwyLNUUsmoecB8TMltAE2JijzgrvvdvePgZWEBC8iss9774XJtH77W7juOli4EHr0iDqq9JFMQl8EdDGzjmZWDxgJzClW5nlgEICZtSA0waypzEBFJHXt3Rt6r/TrF2ZKfOWVsN2gQdSRpZcyE7q7FwBXA/OAFcAsd19mZlPM7NxYsXnAFjNbDswH/sPdt1RV0CKSOjZsgCFDQn/ywYPDVLc/+lHUUaUncy/eHF49srKyPCcnJ5LXFpGqt2xZqIXPmBEGBt13X1g2Um3lFWNmi909K9ExzeUiIpXGHf72N7j33tCs0rBhWITixhuhc+eoo0t/SugiUmHffgtPPx0S+fvvQ6tWYY3Pyy+HFi2iji5zKKGLyEHbsgUefjj0Wtm0KUyoNX06jBoF9etHHV3mUUIXkXL76KPQJv7EE6Ff+eDBoVnl9NPVRh4lJXQRSYo7/P3voVnlxRdDDXzsWLj+eujePeroBDR9rqSR118P7bXDhoV5tPfsiTqi9LB7N2RnhwFBgwbBP/8Jv/oVrFsHjz2mZF6TKKFLWnj//ZDImzSBt9+GM88MvSp+/Wv47LOoo0tNX3wBd90FHTuGmviOHWH5t/Xr4ZZbNO9KTaSELinvk0/CwJXGjeGNNyAvD2bOhE6d4OaboW1bGDEidKfbuzfqaGu+NWtgwoTwuU2cCEcfHSbOWrYsLNDcsGHUEUpJlNAlpX3xRbght317aGZp2xbq1YMLLoDXXgurxU+YAPPnww9+AEcdFWqdmuyzKPdwZfOTn0CXLqHnyk9+Au++G74IzzxTqwalAv2JJGV98w0MHQqrV8Pzz4cuc8UddRTcc08Yfj5jRliXcuJEaNMmdK17/fWQzDJVQQHMmgUnnggnnxy++CZOhLVrQw8WrRiUWpTQJSXt2RPadd98E/73f8PNutI0aABjxoQmmWXL4IorwkjGQYOga9fQBe/f/66e2GuCL78M7/l73wtXM1u2wNSpofnq17/WgsypSgldUo57mHr12WdDUrrggvI9v1s3eOCBUGufPj0sRnzDDSGJXXQR/OMf6VtrX78+LPXWtm14z+3ahaubDz+En/8cDjkk6gilIpTQJeXcfXcYmXjjjSGxH6xGjeCSS0I3vKVLYdy4kNz69w8r6Tz0EGzdWmlhR2rRotDE1KkT3H9/WFxi4cJwxTJ0KNSuHXWEUhmU0CWlPPlkaOMdNSrc3KwsvXqFJoeNG+H3vw9NNNdeG2rtP/0pLFiQerX2b78NX1CnnBLmIZ87NwwCWrMGnnoKjjsu6gilsmn6XEkZ//d/obfFqaeG5FTVc4UsXgyPPhqS39dfhxuEl18e2uKbNKna107Wjh0hQa9efeDP+vXhS6h9+3AlM25czYlbDl5p0+cqoUtKePfdkMg7dQrNBE2bVt9rf/llSOqPPBKWUGvcGEaPDsm9b9+qf/3t2+Ff/0qctDdsKFq2RYtwo7Pwp0+f8CVYR5N8pA0ldElpH38cutXVrx/au6PqgeEe2p0feQSeeSZMSnXccSGxjxxZsRuKX3yROGGvXg2ff160bKtWRZN24U/nztDsgKXZJd0ooUvK2rw59I/Ozw+9T7p2jTqiYOvW0J7/6KOhG+Shh4ZulJdfHm6oFuce3ktJSbt4l8k2bUpO2o0bV897lJpJCV1S0o4dYTrWpUvDaMWTT446ogO5hy+aRx+FP/0p3Ig88UQ477wDE/iXX+5/Xq1aoW07PlEXPu7UScPrpWRK6JJyCgpg+HD4y19Cf/Mf/zjqiMq2ZUsYXfnoo2G+8Dp1wsRWiWraHTqEKQpEyqvCa4qa2WDgAaA28Ji731Hs+CXA3UDhLZrfuvtjBx2xZDT3MMjlxRdDV8JUSOYAzZuHwTrXXx+6Px5+uG5GSvUq85+bmdUGpgI/APKARWY2x92XFyv6jLtfXQUxSoa57bbQF/ymm0JiTzVm0Lp11FFIJkpmYFE/YLW7r3H3XcBMYGjVhiWZ6g9/gF/+MgzBv+22qKMRSS3JJPTWwCdx23mxfcX9xMxyzWy2mbVNdCIzG29mOWaWk6/5S6WYl14KvUR+9KOwEo7WphQpn2QSeqL/VsXvpL4IdHD3nsDfgCcSncjdp7l7lrtntWzZsnyRxixaFAZ1fP31QT1daqiFC+H888NozD/9CerWjToikdSTTELPA+Jr3G2AjfEF3H2Lu38b2/w9cGzlhHeg994Lq9EMGBBuPEnqW7UqTBZ1+OGhlq7h6SIHJ5mEvgjoYmYdzaweMBKYE1/AzI6I2zwXWFF5IRZ12WXwwgthus/jjw8JXlLXpk1hxSGAefO0TqVIRZSZ0N29ALgamEdI1LPcfZmZTTGzc2PFrjWzZWb2HnAtcElVBQxwzjlhYQP3MNXp3LlV+WpSVb76KtTMP/ss9Dfv0iXqiERSW0oPLNqwIST3994LCxZcrU6TKWP37vC3+9vfwhXXWWdFHZFIaihtYFFKz4feunWYee/ss+Gaa8L81Xv2RB2VlMU9rB4/b16Y6ErJXKRypHRChzBR0Z//HEboPfRQWH1l+/aoo5LSTJ4chsjfcku4JyIilSPlEzqE5bN+8xv43e/Cwr+nnAJ5eVFHJYn87ndhEeKf/SwMIBKRypMWCb3QlVeGm2tr1oQltxYvjjoiiffcc+E+x9lnh8SugUMilSutEjqELnD/+EcYmHLqqeGGm0TvH/8IA8L69QvjCDRplUjlS7uEDnDMMWFR3+7dw0x9996begv8ppMVK0KPlrZtwwyKFVnZR0RKlpYJHcIyXa+/HubUvvHGMGtfQUHUUWWejRvDVVO9euH+xkHO+CAiSUjbhA7QqBHMmgWTJu3vHrdtW9RRZY5t22DIkLC82ty5YSUeEak6aZ3QISz19T//E2bve+21sIzZ2rVRR5X+vv02NHctXx5WHOrbN+qIRNJf2if0QuPGhUv+vLwwB8yCBVFHlL727oVLL4X588P85j/8YdQRiWSGjEnoEBYc/uc/w025gQNh9uyoI0pP//mf8PTT4crooouijkYkc2RUQgfo2jXUzvv2DSuz33mnesBUpvvuC4O8rroKJk6MOhqRzJJxCR1CT4tXX4WRI8MN05/9DHbtijqq1PfMM2EKhuHDw2RpGjgkUr0ydnhHgwbw1FNw1FEwZQp8/HFogjnssKgjSz1794YbzhddFKYznjEjTMcgItUrYxM6hBrkrbdC585hkqiTTgor5mRy9zp32LED8vNh8+aiPyXt27IlJPWuXcPI3IYNo34XIpkpoxN6oYsugg4dQje7448PSemkk6KOqnLs2hUSbllJOX77m28Sn6t2bWjeHFq0CM1WXbuGidAKt0eOhO98p3rfn4jsp4Qec+qp8M47YfDRaafB9OkwalTUUZXuk0/g7bdDv/qSEvWXX5b8/GbNQjJu0SIMy+/TZ/924U/LlvsfN20a+vWLSM2khB6nS5fQrXH48DCR1OrVYe7umnBzb/duWLo0JPDCn/gpghs1KpqIO3cumoyLJ+jvfCdMYCYi6UMJvZjmzeGvf4Xx48N83atWwe9/D/XrV28cmzeHL5fC5L1oEezcGY61bx+aOk46KfwcfXRI6CKS2ZTQE6hfH/74x9ADZvLk0KTx3HMh2QNkZ8PNN8P69dCuHdx+O4wZc/Cvt3dvmJEwvvb90UfhWN26oc/8FVeE5H3iiWHpPRGR4pJK6GY2GHgAqA085u53lFBuBPAn4Dh3r9gK0BEzC0m7c2e45BI44YTQA2bRolB737EjlFu3LmxD8kl9+3ZYuHB/8v7nP/dPGtayZUjc48aF38ceq14jIpIc8zKGSZpZbeAj4AdAHrAIGOXuy4uVawK8BNQDri4roWdlZXlOTmrk/LffDmuV7t0basybNh1Ypn37xJN+uYf98bXv3NxwLrMwZ3th08nJJ4cvkJrQZi8iNZOZLXb3rETHkqmh9wNWu/ua2MlmAkOB5cXK/T/gLuAXFYi1RjrppDBdwFlnwYcfJi6zfn34/e23sGRJ0QT+2WfhWOPGoaY/eXI45/HHh54mIiKVIZmE3hr4JG47Dzg+voCZ9QHauvtfzKzEhG5m44HxAO3atSt/tBHq1Ck0jRxxROJ+2k2ahBp2Ts7+aQQ6dYIzzthfA+/RQyMoRaTqJJPQEzUA7GunMbNawH3AJWWdyN2nAdMgNLkkF2LN0axZWChj3DjYs6foscI29Wuv3X/zslWr6o9RRDJXMgk9D2gbt90G2Bi33QToAbxuofG3FTDHzM5N9RujiVx8cahlT5gQVuJp1iw8/q//qv6ujSIi8ZJJ6IuALmbWEdgAjARGFx50921Ai8JtM3sd+EU6JvNCY8eGHxGRmqTMgdzuXgBcDcwDVgCz3H2ZmU0xs3OrOkAREUlOUv3Q3X0uMLfYvl+WUHZgxcMSEZHy0lRLIiJpQgldRCRNKKGLiKQJJXQRkTShhC4ikiaU0EVE0oQSuohImlBCFxFJE0roIiJpQgldRCRNKKGLiKQJJXQRkTShhC4ikiaU0EVE0oQSuohImlBCFxFJE0roIiJpQgldRCRNKKGLiKSJpBK6mQ02s5VmttrMJiU4foWZvW9mS83sLTPrVvmhiohIacpM6GZWG5gKDAG6AaMSJOyn3P0Yd+8N3AXcW+mRiohIqZKpofcDVrv7GnffBcwEhsYXcPcv4zYPAbzyQhQRkWQkk9BbA5/EbefF9hVhZleZ2b8INfRrKyc8KU12NnToALVqhd/Z2VFHJCJRSiahW4J9B9TA3X2qu3cGJgKTE57IbLyZ5ZhZTn5+fvkilSKys2H8eFi3DtzD7/HjldRFMlkyCT0PaBu33QbYWEr5mcCwRAfcfZq7Z7l7VsuWLZOPUg5w882wY0fRfTt2hP0ikpmSSeiLgC5m1tHM6gEjgTnxBcysS9zmWcCqygtRElm/vnz7RST91SmrgLsXmNnVwDygNvC4uy8zsylAjrvPAa42szOA3cAXwMVVGbRAu3ahmSXRfhHJTGUmdAB3nwvMLbbvl3GPJ1RyXFKG228PbebxzS6NGoX9IpKZNFI0RY0ZA9OmQfv2YBZ+T5sW9otIZkqqhi4105gxSuAisp9q6CIiaUIJXUQkTSihi4ikCSV0EZE0oYQuIpImlNBFRNKEErqISJpQQhcRSRNK6CIiaUIJXSpMC22I1Awa+i8VUrjQRuEkYYULbYCmJRCpbqqhS4VooQ2RmkMJXSpEC22I1BxK6FIhJS2ooYU2RKqfErpUyO23h4U14mmhDZFoKKFLhWihDZGaQ71cpMK00IZIzaAauohImkgqoZvZYDNbaWarzWxSguM3mNlyM8s1s1fNrH3lhyoiIqUpM6GbWW1gKjAE6AaMMrNuxYq9C2S5e09gNnBXZQcqIiKlS6aG3g9Y7e5r3H0XMBMYGl/A3ee7e+HwkneANpUbpoiIlCWZhN4a+CRuOy+2ryTjgJcTHTCz8WaWY2Y5+fn5yUcpIiJlSiahW4J9nrCg2VggC7g70XF3n+buWe6e1bJly+SjFBGRMiXTbTEPaBu33QbYWLyQmZ0B3AwMcPdvKyc8ERFJVjI19EVAFzPraGb1gJHAnPgCZtYHeBQ4190/r/wwRUSkLGUmdHcvAK4G5gErgFnuvszMppjZubFidwONgT+Z2VIzm1PC6UREpIokNVLU3ecCc4vt+2Xc4zMqOS4RESknjRQVEUkTSugiImlCCV1EJE0ooYuIpAkldBGRNKGELmkjOxs6dIBatcLv7OyoIxKpXlrgQtJCdjaMHw87YlPErVsXtkGLb0jmUA1d0sLNN+9P5oV27Aj7RTKFErqkhfXry7dfJB0poUtaaNeufPtF0pESuqSF22+HRo2K7mvUKOwXyRRK6JIWxoyBadOgfXswC7+nTdMNUcks6uUiaWPMGCVwyWyqoYuIpAkldBGRNKGELiKSJpTQRUTShBK6iEiaqFG9XHbv3k1eXh7ffPNN1KFIEho0aECbNm2oW7du1KGICDUsoefl5dGkSRM6dOiAmUUdjpTC3dmyZQt5eXl07Ngx6nBEhCSbXMxssJmtNLPVZjYpwfFTzWyJmRWY2YiDDeabb76hefPmSuYpwMxo3ry5rqaK0RS+EqUyE7qZ1QamAkOAbsAoM+tWrNh64BLgqYoGpGSeOvS3KqpwCt9168B9/xS+SupSXZKpofcDVrv7GnffBcwEhsYXcPe17p4L7K2CGEVSgqbwlaglk9BbA5/EbefF9pWbmY03sxwzy8nPzz+YUxRR2Ze3W7ZsoXfv3vTu3ZtWrVrRunXrfdu7du1K6hyXXnopK1euLLXM1KlTya6kalv//v1ZunRppZxLKkZT+ErUkrkpmui62g/mxdx9GjANICsr66DOUagqVqhp3rz5vuR4yy230LhxY37xi18UKePuuDu1aiX+Lpw+fXqZr3PVVVcdXIBSo7VrF/4dJtovUh2SqaHnAW3jttsAG6smnORV5+Xt6tWr6dGjB1dccQV9+/bl008/Zfz48WRlZdG9e3emTJmyr2xhjbmgoIBmzZoxadIkevXqxYknnsjnn38OwOTJk7n//vv3lZ80aRL9+vXj+9//Pm+//TYAX3/9NT/5yU/o1asXo0aNIisrq8ya+IwZMzjmmGPo0aMHN910EwAFBQVceOGF+/Y/+OCDANx3331069aNXr16MXbs2Er/zDKRpvCVqCVTQ18EdDGzjsAGYCQwukqjSkJ1X94uX76c6dOn88gjjwBwxx138J3vfIeCggIGDRrEiBEj6Nat6L3ibdu2MWDAAO644w5uuOEGHn/8cSZNOvKPQtAAAAyqSURBVKCTEO7OwoULmTNnDlOmTOGVV17hoYceolWrVjz77LO899579O3bt9T48vLymDx5Mjk5OTRt2pQzzjiDv/zlL7Rs2ZLNmzfz/vvvA7B161YA7rrrLtatW0e9evX27ZOKKbwyvPnm8O+wXbuQzDUDpFSXMmvo7l4AXA3MA1YAs9x9mZlNMbNzAczsODPLA84DHjWzZVUZNFT/CjWdO3fmuOOO27f99NNP07dvX/r27cuKFStYvnz5Ac9p2LAhQ4YMAeDYY49l7dq1Cc89fPjwA8q89dZbjBw5EoBevXrRvXv3UuNbsGABp512Gi1atKBu3bqMHj2aN954g+9973usXLmSCRMmMG/ePJo2bQpA9+7dGTt2LNnZ2RoYVInGjIG1a2Hv3vBbyVyqU1L90N19rrsf5e6d3f322L5fuvuc2ONF7t7G3Q9x9+buXnr2qQTVfXl7yCGH7Hu8atUqHnjgAV577TVyc3MZPHhwwv7Y9erV2/e4du3aFBQUJDx3/fr1DyjjXr5bDCWVb968Obm5ufTv358HH3yQyy+/HIB58+ZxxRVXsHDhQrKystizZ0+5Xk9Eap6UncslyhVqvvzyS5o0acKhhx7Kp59+yrx58yr9Nfr378+sWbMAeP/99xNeAcQ74YQTmD9/Plu2bKGgoICZM2cyYMAA8vPzcXfOO+88br31VpYsWcKePXvIy8vjtNNO4+677yY/P58dxW9ISErTAKfMVKOG/pdXVCvU9O3bl27dutGjRw86derEySefXOmvcc0113DRRRfRs2dP+vbtS48ePfY1lyTSpk0bpkyZwsCBA3F3zjnnHM466yyWLFnCuHHjcHfMjDvvvJOCggJGjx7N9u3b2bt3LxMnTqRJkyaV/h4kGlXRA0xSg5X30r6yZGVleU5OTpF9K1asoGvXrpHEU9MUFBRQUFBAgwYNWLVqFT/84Q9ZtWoVderUrO9g/c1qng4dEnefbN8+tOtLajOzxe6elehYzcoOss9XX33F6aefTkFBAe7Oo48+WuOSudRMGuCUuZQhaqhmzZqxePHiqMOQFKQBTpkrZW+KikhiGuCUuZTQRdJMlD3AJFpqchFJQ1H1AJNoqYYuIpImlNDjDBw48IBBQvfffz8///nPS31e48aNAdi4cSMjRiResGngwIEU76ZZ3P33319kgM+ZZ55ZKfOs3HLLLdxzzz0VPo9IeWmAU/VSQo8zatQoZs6cWWTfzJkzGTVqVFLPP/LII5k9e/ZBv37xhD537lyaNWt20OcTiZJWcKp+NbYN/brroLLXbejdG2Kz1iY0YsQIJk+ezLfffkv9+vVZu3YtGzdupH///nz11VcMHTqUL774gt27d3PbbbcxdGiRhZtYu3YtZ599Nh988AE7d+7k0ksvZfny5XTt2pWdO3fuK3fllVeyaNEidu7cyYgRI7j11lt58MEH2bhxI4MGDaJFixbMnz+fDh06kJOTQ4sWLbj33nt5/PHHAbjsssu47rrrWLt2LUOGDKF///68/fbbtG7dmhdeeIGGDRuW+B6XLl3KFVdcwY4dO+jcuTOPP/44hx12GA8++CCPPPIIderUoVu3bsycOZO///3vTJgwAQjLzb3xxhsaUSpJK22Ka7XvVw3V0OM0b96cfv368corrwChdn7BBRdgZjRo0IDnnnuOJUuWMH/+fG688cZSJ9B6+OGHadSoEbm5udx8881F+pTffvvt5OTkkJuby9///ndyc3O59tprOfLII5k/fz7z588vcq7Fixczffp0FixYwDvvvMPvf/973n33XSBMFHbVVVexbNkymjVrxrPPPlvqe7zooou48847yc3N5ZhjjuHWW28FwnTA7777Lrm5ufumCL7nnnuYOnUqS5cu5c033yz1i0KkOA1wqn41toZeWk26KhU2uwwdOpSZM2fuqxW7OzfddBNvvPEGtWrVYsOGDWzatIlWrVolPM8bb7zBtddeC0DPnj3p2bPnvmOzZs1i2rRpFBQU8Omnn7J8+fIix4t76623+PGPf7xvxsfhw4fz5ptvcu6559KxY0d69+4NlD5FL4T52bdu3cqAAQMAuPjiiznvvPP2xThmzBiGDRvGsGHDADj55JO54YYbGDNmDMOHD6dNmzbJfIQiQM0a4JSdnRnz1KuGXsywYcN49dVXWbJkCTt37ty3sER2djb5+fksXryYpUuXcvjhhyecMjee2YGr93388cfcc889vPrqq+Tm5nLWWWeVeZ7SrgQKp96F0qfoLctLL73EVVddxeLFizn22GMpKChg0qRJPPbYY+zcuZMTTjiBDz/88KDOLZmppgxwqklt+VV9k1gJvZjGjRszcOBAfvrTnxa5Gbpt2za++93vUrduXebPn8+6RFWPOKeeeuq+haA/+OADcnNzgTD17iGHHELTpk3ZtGkTL7/88r7nNGnShO3btyc81/PPP8+OHTv4+uuvee655zjllFPK/d6aNm3KYYcdxptvvgnAk08+yYABA9i7dy+ffPIJgwYN4q677mLr1q189dVX/Otf/+KYY45h4sSJZGVlKaFLudSUAU7VuVxlaarji6XGNrlEadSoUQwfPrxIj5cxY8ZwzjnnkJWVRe/evTn66KNLPceVV17JpZdeSs+ePenduzf9+vUDwupDffr0oXv37gdMvTt+/HiGDBnCEUccUaQdvW/fvlxyySX7znHZZZfRp0+fUptXSvLEE0/suynaqVMnpk+fzp49exg7dizbtm3D3bn++utp1qwZ//3f/838+fOpXbs23bp127f6kkiyasIAp5rSll8dN4k1fa5UiP5mUtPVlOmEa9UKNfPizMKShckqbfpcNbmISFqrKW351bEOshK6iKS1mtKWXx1fLEkldDMbbGYrzWy1mU1KcLy+mT0TO77AzDocbEBRNQFJ+elvJalizJjQvLJ3b/gdRbt+dXyxlJnQzaw2MBUYAnQDRplZt2LFxgFfuPv3gPuAOw8mmAYNGrBlyxYlihTg7mzZsoUGDRpEHYpIyqjqL5Zkern0A1a7+xoAM5sJDAXil6EfCtwSezwb+K2ZmZczM7dp04a8vDzy8/PL8zSJSIMGDTTYSKQGSSahtwY+idvOA44vqYy7F5jZNqA5sDm+kJmNB8YDtEtwJ6Bu3bp07Ngx2dhFRCROMm3oBw53hOI172TK4O7T3D3L3bNatmyZTHwiIpKkZBJ6HtA2brsNsLGkMmZWB2gK/LsyAhQRkeQkk9AXAV3MrKOZ1QNGAnOKlZkDXBx7PAJ4rbzt5yIiUjFJjRQ1szOB+4HawOPufruZTQFy3H2OmTUAngT6EGrmIwtvopZyznyg9AlRar4WFLtPkOH0eeynz6IofR5FVeTzaO/uCdusIxv6nw7MLKekIbiZSJ/HfvositLnUVRVfR4aKSoikiaU0EVE0oQSesVMizqAGkafx376LIrS51FUlXweakMXEUkTqqGLiKQJJXQRkTShhH4QzKytmc03sxVmtszMJkQdU9TMrLaZvWtmf4k6lqiZWTMzm21mH8b+jZwYdUxRMrPrY/9PPjCzp2PjVjKCmT1uZp+b2Qdx+75jZv9nZqtivw+rrNdTQj84BcCN7t4VOAG4KsGUwplmArAi6iBqiAeAV9z9aKAXGfy5mFlr4Fogy917EAYnjow2qmr1R2BwsX2TgFfdvQvwamy7UiihHwR3/9Tdl8Qebyf8h20dbVTRMbM2wFnAY1HHEjUzOxQ4FfgDgLvvcvet0UYVuTpAw9g8T404cC6otOXub3DgvFZDgSdij58AhlXW6ymhV1BsdaY+wIJoI4nU/cB/AuVY6jZtdQLygemxJqjHzOyQqIOKirtvAO4B1gOfAtvc/a/RRhW5w939UwiVQ+C7lXViJfQKMLPGwLPAde7+ZdTxRMHMzgY+d/fFUcdSQ9QB+gIPu3sf4Gsq8ZI61cTah4cCHYEjgUPMbGy0UaUvJfSDZGZ1Cck8293/HHU8EToZONfM1gIzgdPMbEa0IUUqD8hz98IrttmEBJ+pzgA+dvd8d98N/Bk4KeKYorbJzI4AiP3+vLJOrIR+EMzMCG2kK9z93qjjiZK7/5e7t3H3DoSbXa+5e8bWwNz9M+ATM/t+bNfpFF2uMdOsB04ws0ax/zenk8E3iWPipxu/GHihsk6czBJ0cqCTgQuB981saWzfTe4+N8KYpOa4BsiOrR+wBrg04ngi4+4LzGw2sITQO+xdMmgaADN7GhgItDCzPOBXwB3ALDMbR/jCO6/SXk9D/0VE0oOaXERE0oQSuohImlBCFxFJE0roIiJpQgldRCRNKKGLiKQJJXQRkTTx/wEBApBT3c+/iQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "epochs = range(1, len(acc) + 1)\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "plt.figure()\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a reminder, in chapter 3, the first naive approach to this dataset got you to a test\n",
    "accuracy of 88%. Unfortunately, this small recurrent network doesn’t perform well\n",
    "compared to this baseline (only 85% validation accuracy). Part of the problem is that\n",
    "your inputs only consider the first 500 words, rather than full sequences—hence, the\n",
    "RNN has access to less information than the earlier baseline model. The remainder of\n",
    "the problem is that SimpleRNN isn’t good at processing long sequences, such as text.\n",
    "Other types of recurrent layers perform much better. Let’s look at some moreadvanced layers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_gpu",
   "language": "python",
   "name": "conda_gpu"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
